9 research outputs found
FDive: Learning Relevance Models using Pattern-based Similarity Measures
The detection of interesting patterns in large high-dimensional datasets is
difficult because of their dimensionality and pattern complexity. Therefore,
analysts require automated support for the extraction of relevant patterns. In
this paper, we present FDive, a visual active learning system that helps to
create visually explorable relevance models, assisted by learning a
pattern-based similarity. We use a small set of user-provided labels to rank
similarity measures, consisting of feature descriptor and distance function
combinations, by their ability to distinguish relevant from irrelevant data.
Based on the best-ranked similarity measure, the system calculates an
interactive Self-Organizing Map-based relevance model, which classifies data
according to the cluster affiliation. It also automatically prompts further
relevance feedback to improve its accuracy. Uncertain areas, especially near
the decision boundaries, are highlighted and can be refined by the user. We
evaluate our approach by comparison to state-of-the-art feature selection
techniques and demonstrate the usefulness of our approach by a case study
classifying electron microscopy images of brain cells. The results show that
FDive enhances both the quality and understanding of relevance models and can
thus lead to new insights for brain research.Comment: 12 pages, 7 figures, 2 tables, LaTeX; corrected typo; added DO
CDDFuse: Correlation-Driven Dual-Branch Feature Decomposition for Multi-Modality Image Fusion
Multi-modality (MM) image fusion aims to render fused images that maintain
the merits of different modalities, e.g., functional highlight and detailed
textures. To tackle the challenge in modeling cross-modality features and
decomposing desirable modality-specific and modality-shared features, we
propose a novel Correlation-Driven feature Decomposition Fusion (CDDFuse)
network. Firstly, CDDFuse uses Restormer blocks to extract cross-modality
shallow features. We then introduce a dual-branch Transformer-CNN feature
extractor with Lite Transformer (LT) blocks leveraging long-range attention to
handle low-frequency global features and Invertible Neural Networks (INN)
blocks focusing on extracting high-frequency local information. A
correlation-driven loss is further proposed to make the low-frequency features
correlated while the high-frequency features uncorrelated based on the embedded
information. Then, the LT-based global fusion and INN-based local fusion layers
output the fused image. Extensive experiments demonstrate that our CDDFuse
achieves promising results in multiple fusion tasks, including infrared-visible
image fusion and medical image fusion. We also show that CDDFuse can boost the
performance in downstream infrared-visible semantic segmentation and object
detection in a unified benchmark. The code is available at
https://github.com/Zhaozixiang1228/MMIF-CDDFuse.Comment: Accepted by CVPR 202
Dense 4D nanoscale reconstruction of living brain tissue
Three-dimensional (3D) reconstruction of living brain tissue down to an individual synapse level would create opportunities for decoding the dynamics and structure–function relationships of the brain’s complex and dense information processing network; however, this has been hindered by insufficient 3D resolution, inadequate signal-to-noise ratio and prohibitive light burden in optical imaging, whereas electron microscopy is inherently static. Here we solved these challenges by developing an integrated optical/machine-learning technology, LIONESS (live information-optimized nanoscopy enabling saturated segmentation). This leverages optical modifications to stimulated emission depletion microscopy in comprehensively, extracellularly labeled tissue and previous information on sample structure via machine learning to simultaneously achieve isotropic super-resolution, high signal-to-noise ratio and compatibility with living tissue. This allows dense deep-learning-based instance segmentation and 3D reconstruction at a synapse level, incorporating molecular, activity and morphodynamic information. LIONESS opens up avenues for studying the dynamic functional (nano-)architecture of living brain tissue
NTIRE 2022 Spectral Recovery Challenge and Data Set
This paper reviews the third biennial challenge on spectral reconstruction from RGB images, i.e., the recovery of whole-scene hyperspectral (HS) information from a 3-channel RGB image. This challenge presents the "ARAD_1K" data set: a new, larger-than-ever natural hyperspectral image data set containing 1,000 images. Challenge participants were required to recover hyper-spectral information from synthetically generated JPEG-compressed RGB images simulating capture by a known calibrated camera, operating under partially known parameters, in a setting which includes acquisition noise. The challenge was attended by 241 teams, with 60 teams com-peting in the final testing phase, 12 of which provided de-tailed descriptions of their methodology which are included in this report. The performance of these submissions is re-viewed and provided here as a gauge for the current state-of-the-art in spectral reconstruction from natural RGB images
Saturated reconstruction of living brain tissue
Complex wiring between neurons underlies the information-processing network enabling all brain functions, including cognition and memory. For understanding how the network is structured, processes information, and changes over time, comprehensive visualization of the architecture of living brain tissue with its cellular and molecular components would open up major opportunities. However, electron microscopy (EM) provides nanometre-scale resolution required for full in-silico reconstruction1–5, yet is limited to fixed specimens and static representations. Light microscopy allows live observation, with super-resolution approaches6–12 facilitating nanoscale visualization, but comprehensive 3D-reconstruction of living brain tissue has been hindered by tissue photo-burden, photobleaching, insufficient 3D-resolution, and inadequate signal-to-noise ratio (SNR). Here we demonstrate saturated reconstruction of living brain tissue. We developed an integrated imaging and analysis technology, adapting stimulated emission depletion (STED) microscopy6,13 in extracellularly labelled tissue14 for high SNR and near-isotropic resolution. Centrally, a two-stage deep-learning approach leveraged previously obtained information on sample structure to drastically reduce photo-burden and enable automated volumetric reconstruction down to single synapse level. Live reconstruction provides unbiased analysis of tissue architecture across time in relation to functional activity and targeted activation, and contextual understanding of molecular labelling. This adoptable technology will facilitate novel insights into the dynamic functional architecture of living brain tissue